In [20]:
from IPython.display import HTML
import pandas as pd
import datetime

HTML('''<script>
code_show=true; 
function code_toggle() {
 if (code_show){
 $('div.input').hide();
 } else {
 $('div.input').show();
 }
 code_show = !code_show
} 
$( document ).ready(code_toggle);
</script>
<form action="javascript:code_toggle()"><input type="submit" value="Source"></form>''')
Out[20]:

Product Management - Metrics & OKRs

Data retrieved on: 21-09-2019

This document details a process for the acquisition and analysis of data pertaining to Product Management Metrics and OKRs.


Table of Contents

Data

Metrics


Data

Source

The Product Management Metrics & OKRs are derived from two sources of data- issue metadata and parent/child relationships between PM and Engineering JIRA issues. Figure 1.0 details the proposed metadata and relationships.

To support the veracity of the data acquired, it is requested that Product Engineering contribute to the mapping of Engineering issues to Product Management issues.

Acquisition

Raw changelog and issue metadata is retrieved using the Python JIRA API and transformed using Pandas. Figure 2.0 contains a sample from the PM JIRA project dataset.

Figure 1.0

Proposed Metadata and relationship between PM & Engineering JIRA issues.



Figure 2.0

Sample of data extracted from PM → Engineering JIRA issue relationships.

In [21]:
pd.set_option('display.max_columns', None)

sample = pd.read_csv('pm_changelog_clean.csv').drop(['Unnamed: 0', 'remove'], axis=1)
sample[100:105]
Out[21]:
pm_key pm_id related_key related_id related_type updated_time updated_status fix_version labels 5.2_candidate 5.7_candidate 5.8_candidate Cloud_JSX Customer_Priority Design_Implementation Enablement Features JSX JSXSquad Milestone1 Milestone2 Perf_Lists Performance_Squad Platform Platforms Product_Performance Product_Solutions Q4_2019_candidate_feature RNCloud RN_Upgrade Services_Priority debugging dep_core installer jira_escalated
100 PM-114 85554 US-10155 85811 Story 2019-07-23T11:12:11.237-0400 Doing [{'self': 'https://youilabs.atlassian.net/rest... ['Milestone2'] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
101 PM-114 85554 US-10155 85811 Story 2019-07-17T16:22:29.749-0400 To Do [{'self': 'https://youilabs.atlassian.net/rest... ['Milestone2'] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
102 PM-109 80104 RN-2076 86631 Epic 2019-08-28T14:33:01.193-0400 Closed [{'self': 'https://youilabs.atlassian.net/rest... ['Milestone1', 'Product_Performance'] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
103 PM-109 80104 RN-2076 86631 Epic 2019-07-29T09:37:01.146-0400 Doing [{'self': 'https://youilabs.atlassian.net/rest... ['Milestone1', 'Product_Performance'] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0
104 PM-109 80104 RN-2076 86631 Epic 2019-07-29T09:36:50.698-0400 To Do [{'self': 'https://youilabs.atlassian.net/rest... ['Milestone1', 'Product_Performance'] 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Metrics

Milestone Investment

The number of issues closed and hours spent towards each of the product investment categories (Product Backlog, CRT, Quality and Engineering) by release and milestone.

Nota Bene: Data Veracity

For each release, a subset of issues do not include metadata (as listed below) that corresponds to Product Backlog, CRT, Quality or Engineering categories. Accordingly, the percentages listed for categories in each release do not have a sum total of 100%.

Data Point Data Source Work Metric Time Metric
Milestone Label: Milestone, Date Range N/A N/A
Release Field: Fix_Version, Date Range N/A N/A
Product Backlog Children of issues linked to PM issue Sum of Closed Issues in milestone label/date range. Sum time deltas of status changes to doing, test, documentation, verified, closed.
CRT Label: Customer_Priority, Services_Priority on PM issue Sum of Closed Issues in milestone label/date range. Sum time deltas of status changes to doing, test, documentation, verified, closed.
Quality Issue Type: Bug Sum of Closed Issues in milestone label/date range. Sum time deltas of status changes to doing, test, documentation, verified, closed.
Engineering Label: DDB, Delta between PM-linked issues and all Engineering issues Sum of Closed Issues in milestone label/date range. Sum time deltas of status changes to doing, test, documentation, verified, closed.



In [24]:
from scripts import metrics_okrs
import importlib
from pandas.api.types import is_string_dtype

importlib.reload(metrics_okrs)

metrics_okrs.milestone_investment('5.7.0')
metrics_okrs.milestone_investment('5.6.0')
metrics_okrs.milestone_investment('5.5.0')

Nota Bene: Data Veracity


For each release, a subset of issues do not include metadata (shown below as Unlabelled) that corresponds to Product Backlog, CRT, Quality or Engineering categories. Accordingly, the percentages listed for categories in each release do not have a sum total of 100%.
CRT Engineering PM Backlog Quality Unlabelled
0 6.94% 15.28% 11.11% 40.28% 26.39%
CRT Engineering PM Backlog Quality Unlabelled
0 5 11 8 29 19
Total Number of Issues: 72

Nota Bene: Data Veracity


For each release, a subset of issues do not include metadata (shown below as Unlabelled) that corresponds to Product Backlog, CRT, Quality or Engineering categories. Accordingly, the percentages listed for categories in each release do not have a sum total of 100%.
CRT Engineering PM Backlog Quality Unlabelled
0 8.12% 13.12% 12.50% 36.25% 30.00%
CRT Engineering PM Backlog Quality Unlabelled
0 13 21 20 58 48
Total Number of Issues: 160

Nota Bene: Data Veracity


For each release, a subset of issues do not include metadata (shown below as Unlabelled) that corresponds to Product Backlog, CRT, Quality or Engineering categories. Accordingly, the percentages listed for categories in each release do not have a sum total of 100%.
CRT Engineering PM Backlog Quality Unlabelled
0 10.48% 8.06% 8.87% 34.68% 37.90%
CRT Engineering PM Backlog Quality Unlabelled
0 13 10 11 43 47
Total Number of Issues: 124

Product Management OKRs

The number of issues closed, open issues remaining and time spent towards product objectives (Strategic Accounts, vJSX, Performance, Cloud JSX, RN Upgrade, Enablement and Platform) by release and milestone.

Data Point Data Source Work Metric Time Metric
Milestone Label: Milestone, Date Range N/A N/A
Release Field: Fix_Version, Date Range N/A N/A
Product Backlog Label: Strategic_Accounts, vJSX, Performance, Cloud JSX, RN Upgrade, Enablement, Platform Sum of closed issues with Product Backlog label in milestone label/date range.

Sum of total issues with Product Backlog label in milestone label/date range.
Sum time deltas of status changes to doing, test, documentation, verified, closed.



In [23]:
importlib.reload(metrics_okrs)
metrics_okrs.pm_okrs()

CRT Breakdown

Forthcoming